Text Correction Using Approaches Based on Markovian Architectural Bias

Several authors have reported interesting results obtained by using untrained randomly initialized recurrent part of an recurrent neural network (RNN). Instead of long, difficult and often unnecessary adaptation process, dynamics based on fixed point attractors can be rich enough for further exploitation for some tasks. The principle explaining untrained RNN state space structure is called Markovian architectural bias [1, 8] and several methods using this behavior were studied. In this paper we apply these approaches to correct corrupted symbols from symbol sequence. These approaches share some properties with variable length Markov models hence our experiments are inspired by the paper dealing with the text correction on the bible dataset.